Skip to content

1.5.91: scene backdrops, transcript views, stages that rehearse and record - #139

Open
echarles wants to merge 282 commits into
mainfrom
feat/scenes
Open

echarles wants to merge 282 commits into
mainfrom
feat/scenes

Conversation

@echarles

Copy link
Copy Markdown
Member

Scenes and stages for 1.5.91.

  • Scene backdrops (studio, hero, plain) for A2ATeamGraph, read from stage.backdrop.
  • Transcript views: lines, a table and a mermaid sequence diagram, with a drill-down and a JSON export.
  • loop scenes rehearse --transcript and loop scenes record; recordings generated into specs/sceneRecordings and played with useSceneRecording.
  • A coding agent's Shield steps as transcript lines, with the Shield's decision in the drill-down.
  • A nightly rehearsal workflow that does nothing without the demo account's key.
  • The sessions plugin's circular import is cut.
  • Docs: Scenes and Stages pages.

Needs agentspecs 0.0.137.

Copilot AI balanced review requested due to automatic review settings October 29, 2026 09:23

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved defects can corrupt scene YAML or update the wrong spec file, while stale imports prevent budget tests from collecting.

10 open issues
What changed in this PR

This PR extends Agent Runtimes’ scene presentation and recording support for 1.5.91, adds a Shield-backed coding-agent scene, and refactors Python imports to break the sessions plugin’s circular dependency.

Changes:

  • Adds backdrop infrastructure, transcript views, drill-downs, exports, and graph selection integration.
  • Adds rehearsal transcript export, recording storage and playback, and updated scene catalogues.
  • Updates documentation, regression coverage, and delegation/output-format imports.
File Description
src/​types/​scenes.ts Defines recording file, beat, and line types.
src/​specs/​scenes.ts Adds the coding-agent scene and hero defaults.
src/​specs/​sceneRecordings.ts Adds the generated recording catalogue.
src/​specs/​actions.ts Registers Code Explainer action mappings.
src/​specs/​__tests__/​scenes.test.ts Tests scene, backdrop, and recording contracts.
src/​examples/​ScenesExample.tsx Connects backdrops and transcript-driven edge selection.
src/​examples/​__tests__/​scenesExample.test.tsx Updates runtime-member and port expectations.
src/​components/​teams/​transcript/​useSceneRecording.ts Adds timed playback and controls.
src/​components/​teams/​transcript/​TranscriptViewsTypes.ts Defines shared transcript-view types.
src/​components/​teams/​transcript/​TranscriptTable.tsx Adds selectable, sortable transcript rows.
src/​components/​teams/​transcript/​transcriptRows.ts Derives row metadata, filters, and Markdown.
src/​components/​teams/​transcript/​transcriptExport.ts Adds JSON serialization, parsing, and downloads.
src/​components/​teams/​transcript/​TranscriptDetail.tsx Renders span details and Shield decisions.
src/​components/​teams/​transcript/​sequenceDiagramRules.ts Generates Mermaid sequence diagrams.
src/​components/​teams/​transcript/​selection.ts Maps transcript selections to graph edges.
src/​components/​teams/​transcript/​recordingPlayback.ts Computes playback timing and graph activity.
src/​components/​teams/​transcript/​RecordingLabel.tsx Labels recordings with their capture date.
src/​components/​teams/​transcript/​index.ts Exports transcript and playback APIs.
src/​components/​teams/​transcript/​__tests__/​transcriptRows.test.ts Tests rows, filtering, sorting, and Markdown.
src/​components/​teams/​transcript/​__tests__/​transcriptExport.test.ts Tests JSON round-trips and validation.
src/​components/​teams/​transcript/​__tests__/​transcriptDetailRules.test.ts Tests span and Shield detail extraction.
src/​components/​teams/​transcript/​__tests__/​sequenceDiagram.test.ts Tests diagram generation and line mappings.
src/​components/​teams/​transcript/​__tests__/​selection.test.ts Tests edge-selection mapping.
src/​components/​teams/​transcript/​__tests__/​recordingPlayback.test.tsx Tests playback timing, activity, and controls.
src/​components/​teams/​transcript/​__tests__/​lazyMermaid.test.ts Checks that Mermaid loads dynamically.
src/​components/​teams/​transcript/​__tests__/​golden/​sales-and-accounting.mmd Captures the successful exchange diagram.
src/​components/​teams/​transcript/​__tests__/​golden/​open-and-failed.mmd Captures open and failed exchanges.
src/​components/​teams/​transcript/​__tests__/​fixture.ts Shares transcript fixtures across view tests.
src/​components/​teams/​SceneView.tsx Integrates transcript views and Inspector access.
src/​components/​teams/​sceneTranscriptRules.ts Adds Shield steps and completion timestamps.
src/​components/​teams/​SceneTranscript.tsx Adds line selection and highlighting.
src/​components/​teams/​index.ts Exports backdrop and transcript modules.
src/​components/​teams/​backdrops/​types.ts Defines backdrop contracts and graph data.
src/​components/​teams/​backdrops/​theme.ts Applies light and dark backdrop styling.
src/​components/​teams/​backdrops/​studio.tsx Extracts studio rendering and selected-edge halos.
src/​components/​teams/​backdrops/​plain.tsx Adds static, print-oriented graph rendering.
src/​components/​teams/​backdrops/​index.ts Registers backdrops and fallback lookup.
src/​components/​teams/​backdrops/​halo.tsx Renders selected-edge highlights.
src/​components/​teams/​backdrops/​faces.tsx Creates framed member faces.
src/​components/​teams/​backdrops/​edges.ts Shares edge and arrow geometry.
src/​components/​teams/​backdrops/​catalogue.ts Defines backdrop choices and scene lookup.
scripts/​codegen/​generate_scenes.py Generates the separate recording catalogue.
package.json Bumps the release and adds Mermaid.
examples/​sales-accounting-a2a/​README.md Documents the additional runtime member.
docs/​sidebars.js Adds Scenes documentation navigation.
docs/​docs/​scenes/​stages.mdx Documents stages, rehearsals, and recordings.
docs/​docs/​scenes/​index.mdx Documents backdrops and transcript inspection.
docs/​docs/​cli/​index.mdx Documents transcript export and recording commands.
docs/​docs/​apps/​teams-over-a2a.mdx Updates local scene port assignments.
CHANGELOG.md Summarizes the 1.5.91 release.
agent_runtimes/​types/​types.py Adds the Python stage backdrop field.
agent_runtimes/​transports/​a2a.py Uses relocated delegation helpers.
agent_runtimes/​tests/​test_subagents_a2a.py Updates the refusal-helper import.
agent_runtimes/​tests/​test_sessions_plugin_import_order.py Adds cold-import plugin regression coverage.
agent_runtimes/​tests/​test_serve_scenes.py Updates runtime-member and port assertions.
agent_runtimes/​tests/​test_scenes_transcript.py Tests browser-exported transcript compatibility.
agent_runtimes/​tests/​test_scenes_specs.py Tests the expanded catalogue and backdrops.
agent_runtimes/​tests/​test_scenes_rehearse.py Tests rehearsal transcript output.
agent_runtimes/​tests/​test_pap_company_turns.py Updates the delegation-key import.
agent_runtimes/​tests/​test_orchestration_extension_schema.py Updates the delegation-key import.
agent_runtimes/​tests/​orchestration/​bindings.py Updates the refusal-helper import.
agent_runtimes/​tests/​fixtures/​scene_transcript.json Adds allowed and refused Shield steps.
agent_runtimes/​tests/​fixtures/​scene_transcript_export.json Adds the shared JSON export fixture.
agent_runtimes/​specs/​scenes.py Adds the Python coding scene and hero defaults.
agent_runtimes/​specs/​apps.py Adds the Code Explainer application.
agent_runtimes/​specs/​actions.py Registers Python Code Explainer action mappings.
agent_runtimes/​routes/​acp.py Uses relocated delegation helpers.
agent_runtimes/​output/​formats.py Owns the notebook media-type constant.
agent_runtimes/​orchestration/​extension_schema.py Uses the relocated delegation key.
agent_runtimes/​orchestration/​documents.py Imports the shared notebook media type.
agent_runtimes/​orchestration/​budget.py Uses the relocated delegation key.
agent_runtimes/​loop/​scenes/​stage.py Reuses shared output media types.
agent_runtimes/​loop/​scenes/​rehearsal.py Retains and exports played transcript lines.
agent_runtimes/​loop/​scenes/​recording.py Validates, stores, and references recordings.
agent_runtimes/​guardrails/​model_budget.py Removes delegation-envelope dependencies.
agent_runtimes/​context/​delegation.py Owns delegation budget and refusal helpers.
agent_runtimes/​commands/​scenes.py Adds transcript export and the record command.
agent_runtimes/​apps/​visitors.py Uses the relocated delegation key.
agent_runtimes/​apps/​a2a.py Consolidates delegation metadata imports.

Review effort: Balanced

Comment thread agent_runtimes/commands/scenes.py Outdated
console.print(f"[red]✗[/red] {refused}", highlight=False)
raise typer.Exit(1)
folder = specs_folder(module, scene_name, specs_dir)
spec_file = folder / f"{scene.id}.yaml"
return checkpoint_id if isinstance(checkpoint_id, str) and checkpoint_id else None


def delegated_budget(meta: Any) -> dict[str, Any] | None:
Comment thread agent_runtimes/loop/scenes/recording.py Outdated
Comment on lines +120 to +124
block = [
" recording:",
f" path: {path}",
f" taken: {_quoted(taken)}",
*([f" note: {_quoted(note)}"] if note else []),
"count": len(beat.transcript),
}
)
lines.extend(beat.transcript)
Comment on lines +109 to +110
aria-selected={onSelect ? selected === line.key : undefined}
onClick={onSelect ? () => onSelect(line.key) : undefined}
Comment on lines +309 to +310
if (isShieldTurn(span)) {
lines.push(...shieldStepsOf(span, turnMember(span, byId, members)));
<DataTable
aria-labelledby={title}
data={rows}
columns={COLUMNS}
Comment on lines +210 to +211
initialSortColumn="at"
initialSortDirection="ASC"
Comment on lines +133 to +134
setElapsed(total);
setPlaying(false);
Comment thread src/examples/ScenesExample.tsx Outdated
Comment on lines +435 to +439
setSelectedEdge(
line
? (selectedEdgeOf(selectionOfLine(line), transcriptMembers) ??
null)
: null,
…s own shared above and below the members (the scene editor's Preview)
…s/paletteColor: useSpecColor over primer-addons' useColorPalette, which every released primer-addons has), not by usePaletteColor, which no released primer-addons exports yet: the TypeScript build was red (TS2305). A palette name is the palette's value, read by name; a hex is itself; anything else, and a hue a released palette does not have yet, is nothing, so the backdrop draws its own colour. It gives way to usePaletteColor once primer-addons is released and the range raised
….139: usePaletteColor and resolvePaletteColor in place of the local resolver; the specs regenerated, every colour a name
…a glowing circle in its own colour and nothing more, a system a smaller dimmer circle, thin lines with the light that flows along them (space's luminousEdges, without their words), a dark ground in both modes; a click on the circle keeps its details (figureKeeps, aria-expanded on the figure). On every backdrop a member's details open in a card on hover or focus of its figure or name and are kept by a click on its name (aria-expanded, Escape and a second click close it); the figure's click stays its balloon's where a backdrop has one; a system's card peeks on hover everywhere; an open card's node stands over the others
…layer's orb, the design library's SvgAvatarOrb in its theme tone, its three colours the member's own (usePaletteColor; as it is, lighter, deeper), the theme's when it has none; its name and where it runs under it, its details' card on hover and a click as on every backdrop; luminous lines with their words, as on space. The orb is loaded when first drawn, through orbCharacters' lazy SvgAvatarOrb (now exported): @datalayer/design was already a dependency, and no other backdrop loads it. A face carries the member's colour
…ls on every backdrop; agentspecs 0.0.140 and the specs regenerated (the coding agent's recording)
… and drawn on its workspace, start and end one span — the home page's coding agent stage said 'its runtime: read' (30 October 2026)
…epository an application is lent, as the harness's workspace holds it now: its URL and pinned revision, HEAD, whether HEAD is that revision, and the lines of git status --porcelain, never a file's contents; shown as the computer is (a deployment's to its owner and its application's editors, as ai-agents answers; nobody it acts for); 404 when it is lent none, 409 when the workspace is no checkout — what the nightly stage rehearsal reads to know the coding agent left README.md as it was lent
… draft 0.3 no longer reserves (the schema line follows agentspecs)
…o network by default, an environment of what was handed in, no capability, bounded files, CPU, memory and time; a kernel that refuses isolation refuses the run
…lan, execute, normalize_result), each adapter versioned on its own; capabilities native, framework-run or unsupported; a source kept by digest and unpacked read-only; a runtime's host and routes; secrets by name, their values taken out of what comes back; the fake adapter that proves it
…that source, and what a framework left that cannot be read is the task's failure
…e sandbox with no sample run; solvers, tools, scorers, metrics and models classified; one sample per trial through inspect_ai.eval with the launch's pool the only scheduler; the eval log kept whole; every scorer on the task and no aggregate nobody chose; a tool runs only if the launch approved it, by an Inspect approval policy; proved identical to a local pinned run on Inspect's mock model
…ession is a random sid sent as the Session header, the account's access token kept beside it, every operation asked of the person through the SDK and approved by a message in its conversation, an opaque revision, and other terms another operation
…r pap_confirm is made, ask_the_person is gone, company_standing_permission and user_approval_required are removed from the builders and the operation's spec, and an operation names its tool
…t_as, never read, is gone) — agent-runtimes passes key and requires agent-shield >= 0.2.18; what 1.6.2 and 1.6.3 carry
…y-example spec itself, CI having no agentspecs clone; the changelog says what 1.6.4 and 1.6.5 carry
… Bedrock model is made at start and asks for one; AgentCore sets it itself); the build waits up to eight attempts 90 s apart for PyPI's index, which lagged its JSON on 1.6.5
…rvices assume organizations' DatalayerRole as, allowed only sts:AssumeRole on *DatalayerRole* (each role still requiring its external id); its key created by hand, never a stack output
…try carries {target, started_at, ended_at, seconds, microvm_started_at}: when the invocation arrived, when its run ended, the wall time between, and when the runtime's process started
…ried by generate_ops.py (decisionModel in TypeScript), on BattleContestantSpec in the Python and TypeScript types; battles regenerated from agentspecs 0.0.172 (feat/apps-next): in pong and invaders, Red is played by Jev (cloudflare:wrk/typesafe/jev)
… where IAM is (DATALAYER_IAM_URL from the discovery URL), else it refused to start; a SigV4 runtime is not told its ARN, its exchange off
…Core runtime (else every invocation 401); a {"prompt"} session's turns kept in the process and put before its next prompt
… HTTPS with the token, SigV4's refusal said; the CLI sends a prompt so the session keeps its turns
…oAgentSkillSpec), so an agent's page can show each skill's mark
…ts (agentspecs 0.0.170-0.0.171), cogs' identities, Pong's Red played by Jev; the sandbox scaffold writes identity: datalayer-acting-agent, which the specs now require
…on a person's own machine with their DATALAYER_API_KEY remembers on Datalayer (DatalayerMemory, through the runtimes service's memories: the same memories as on a Datalayer runtime, the datalayer-memory skills included); an MCP server without its credentials no longer holds an agent back in the picker, listed in red; a versioned env var reference (NAME:0.0.1) read by its name in the picker and in codemode's server env
…came last — the Studio's AG-UI turns (relayed by ai-agents) then a loop prompt on the same session carry them; with loop agentcore invoke --bearer, the Marketer in the loop picker, a local agent's mem0 memories on Datalayer with DATALAYER_API_KEY, the skills' marks and TAVILY_API_KEY's link
…in the Python and TypeScript types, carried by generate_mcp_servers.py and generate_skills.py (checked, a sentence when malformed), regenerated from agentspecs 4382a36; ${VAR:-default} in an MCP server's args and env (Bluesky's server where BLUESKY_MCP_SERVER_DIR says, else /opt/mcp-servers); PolicySpec carries dogwood, rate_limits and firewall (build:lib refused the Dogwood example policies); the ready line says Jupyter Code Sandbox; a remote server names its headers or its auth
….env_expansion) wherever a server is started — the lifecycle manager, Codemode's registry, the agent factory, codemode's rebuild; Codemode handed Bluesky's ${BLUESKY_MCP_SERVER_DIR:-…} path to node unexpanded
…T_UID) runs the deployment's agent, made at the first invocation as a kept runtime's is (kept_payload, from what ai-agents answers the caller), each invocation through the application's session routes: the session opened by its first run, resumed from its record on a new microVM, recorded under the deployment where it runs ({kind: agentcore, arn}); the contract's routes answered while a deployment is served
…loyment's agent and records its sessions under the deployment, resumed from the record on a new microVM; the release build's policy types (1.6.10 was not published)
…AM's binding page as the return URL and a state IAM issued to the person (identity.consent: kept per person and runtime while it stands, forgotten once used, refused in a sentence without IAM in the path), so IAM binds the consent to who is signed in; the github-repositories tool (the person's repositories with the provider's token, never GITHUB_TOKEN), on the Marketer from agentspecs 64af336

This branch was successfully deployed

2 active (outdated) deployments
pypi — 63ae1628 Deployed Nov 10, 2026 by echarles via pypi #247
npm — 63ae1628 Deployed Nov 10, 2026 by echarles via npm #247
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants